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Abstract 

The Carnegie classification system has served as a framework for research on colleges and 
universities for more than 30 years. Today, the system’s developers are exploring criteria that 
more effectively differentiate among institutions. One approach being considered is classifying 
institutions based on students’ educational experiences. This study explored whether it is 
possible to create a typology of institutions based on students’ experiences. Results indicated 
that such a typology could be created, and the types were somewhat independent of institutional 
mission (i.e., Carnegie classification). 
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A Typology of Student Engagement for American Colleges and Universities 

Since its development in 1973, the Carnegie classification system has served as a framework 
for research on colleges and universities. These mission-oriented classifications were revised in 
2000 to reflect recent changes in higher education, and the system’s developers continue to 
explore criteria that more effectively differentiate among institutions (McCormick, 2000). One 
approach being considered is classifying institutions based on students’ educational experiences 
(McCormick, personal communication, May 21, 2003). 

Examining students’ experiences is important because student engagement in educationally 
purposeful activities has desirable effects on student learning and success during college (Astin, 
1977, 1993; Feldman & Newcomb, 1969; Kuh, Pace, & Vesper, 1997; Pace, 1990; Pascarella & 
Terenzini, 1991). Based on their review of 20 years of research, Pascarella and Terenzini (1991, 
p. 610) concluded, “one of the most inescapable and unequivocal conclusions we can make is 
that the impact of college is largely determined by the individual’s quality of effort and level of 
involvement in both academic and nonacademic activities.” 

Related Literature 

Student-engagement theory had its origin in the work of Astin (1984, 1985), Pace (1984), 
and Kuh and his colleagues (Kuh, Schuh, Whitt, & associates, 1991; Kuh, Whitt, & Strange, 
1989). Although these writers used different terminology to describe their concepts of student 
engagement, their views were based on the simple, but powerful, premise that students learn 
from what they do in college. Research has strongly supported this assumption, indicating that 
engagement is positively related to objective and subjective measures of gains in general abilities 
and critical thinking (Endo & Harpel, 1982; Gellin, 2002; Kuh, Hu, & Vesper, 2000; Kuh & 
Vesper, 1997; Pascarella, Duby, Terenzini, & Iverson, 1983; Pascarella, et al., 1996; Pike, 1999, 
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2000; Pike & Killian, 2001; Pike & Kuh, in press; Pike, Kuh, & Gonyea, 2003; Terenzini, 
Pascarella, & Blimling, 1996). Student engagement is also positively linked to grades (Astin, 
1977, 1993; Indiana University Center for Postsecondary Research, 2002; Pike, Schroeder, & 
Berry, 1997) and persistence rates (Astin, 1985; Pike, Schroeder & Berry, 1997). 

A second important premise of the frameworks of Astin, Kuh, and Pace is that, even though 
the focus is on student engagement, institutional policies and practices influence levels of 
engagement on campus. For example, research has not been able to produce consistent 
relationships between students’ pre-college characteristics (e.g., gender, minority status and 
entering ability levels) and engagement during college (Bauer & Liang, 2003; Endo & Harpel, 
1982; Hu & Kuh, 2002; Indiana University Center for Postsecondary Research, 2002; Iverson, 
Pascarella, & Terenzini, 1984; Kuh, Hu, & Vesper, 2000; Pike, 1999, 2000; Pike & Killian, 

2001; Pike & Kuh, in press; Pike, Kuh, & Gonyea, 2003; Pike, Schroeder, & Berry, 1997). 
Moreover, the strength of those relationships, when present, was quite low. Studies by Pike and 
his colleagues have found that students’ background characteristics generally account for 1% to 
5% of the variance in levels of engagement (Pike 1999, 2000; Pike & Killian, 2001; Pike & Kuh, 
in press, Pike, Kuh, & Gonyea, 2003). 

The influence of institutional characteristics on student engagement extends well beyond 
global characteristics such as size and institutional mission. Although both conventional wisdom 
and research studies suggest that attending small liberal arts colleges is associated with higher 
levels of engagement (Hu & Kuh, 2002; Kuh, 1981; Kuh & Siegel, 2000; Pascarella, Wolniak, 
Cruce, & Blaich, 2004), other studies have come to a different conclusion. For example, Pike, 
Kuh, and Gonyea (2003) found that differences in levels of engagement across Carnegie 
classifications disappeared after taking into account the background characteristics of the 
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students. The most important institutional factors are thought to be the policies and practices 
adopted by institutions to increase student engagement. Several studies have shown that living 
on campus, as opposed to commuting to college, is positively related to engagement (Chickering, 
1975; Pike & Kuh, in press; Terenzini, Pascarella, & Blimling, 1996). The gains associated with 
on-campus living are further enhanced by participating in learning communities, which 
substantially increases student engagement, self-reported gains in learning, and persistence 
(Indiana University Center for Postsecondary Research, 2002; Pike, 1999; Pike, Schroeder, & 
Berry, 1997; Zhao & Kuh, 2004). 

Given the powerful relationship between engagement and positive educational outcomes, it 
is not surprising that Astin (1985, p. 36) argued that “the effectiveness of any educational policy 
or practice is directly related to the capacity of that policy or practice to increase student 
involvement.” Some student engagement surveys are designed to assess the effectiveness of 
these institutional policies and practices (see Kuh, Pace, & Vesper, 1997). The most widely used 
instrument at this time is the National Survey of Student Engagement (NSSE) (Kuh, 2001a, 
2003). 

The NSSE was developed as an alternative to reputation- and resource-based ratings of news 
magazines and college guidebooks. Rather than ranking institutions, data from the NSSE survey, 
The College Student Report, provide colleges and universities with information about the 
activities in which their students engage and point to areas where improvement may be needed. 
Results from the first four years of the survey were generally consistent with previous theory and 
research on student engagement. Students attending small, selective liberal arts colleges tended 
to be somewhat more engaged than their counterparts at large public universities (Indiana 
University Center for Postsecondary Research, 2000, 2001, 2002). However, there was 
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substantial variation in institutions’ student engagement scores within institutional categories as 
represented by Carnegie classification and size. Moreover, institutions with high engagement 
scores in one area generally did not have high scores in all areas (Kuh, 2001a; 2003). 

The findings that institutions with similar characteristics and missions differed substantially 
in both levels and types of engagement raise an important question: 

Is it possible to create a typology of engaging institutions that is independent of the 
traditional Carnegie classifications? 

The answer to this question is important for two reasons. First, as pointed out earlier, extant 
research shows that the Carnegie classification does not reliably distinguish institutions in terms 
of their educational effectiveness as represented by student engagement. Second, being able to 
classify institutions according to levels of student engagement will make it possible to identify 
colleges and universities that may be used as benchmarks by other schools with similar missions 
and other characteristics. For these reasons we set out to determine whether four-year colleges 
and universities could be sorted into meaningful categories based on patterns of student 
engagement. 

Research Methods 

A variety of approaches can be used to generate typologies (see Aldenderfer & Blashfield, 
1984; Kruskal & Wish, 1978; Rummel, 1970). The approach selected for the present research 
was Q factor analysis (Burt 1937; Stephenson, 1953). According to Cattell (1952, p. 101), “Q 
technique is most useful if one wishes immediately to see how many types there are in a 
population and to divide it up into types.” One advantage of Q factor analysis over cluster 
analysis is that institutions can belong to more than one engagement type (Gorsuch, 1983). An 
important practical advantage of Q factor analysis, as opposed to multidimensional scaling, is 
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that commercially available factor analysis programs can accommodate very large data sets 
(Kruskal & Wish, 1978). 

Participating Institutions 

The data for this study came from the 2001 administration of The College Student Report . 
The NSSE 2001 respondents included 177,103 first-year and senior students who were randomly 
selected from the populations of 321 participating colleges and universities. Students at 261 
institutions had the option of responding either via a paper-and-pencil questionnaire or via the 
Web. Sixty schools opted for web-only administration of the survey. Of the 321 institutions, 4 
were excluded from the study because of specialized missions and/or very low numbers of 
respondents. Table 1 displays the characteristics of the NSSE 2001 institutions and a national 
profile of all four-year colleges. These data show that the NSSE institutions were very similar to 
the national profile in terms of geographic region and urban-rural location. Baccalaureate- 
General colleges were underrepresented among the NSSE participants, whereas 
Doctoral/Research-Extensive and Baccalaureate Liberal Arts institutions were overrepresented 
among the NSSE participants. 



Insert Table 1 about here 



Only seniors were included in the current study for two reasons. First, they have had a 
wider range of experiences during college and arguably can provide more informed reports about 
a variety of college activities. Second, the experiences of first-year students and seniors differ 
substantially in terms of curriculum (coursework for first-year students emphasizes general 
education, while seniors are concentrated in the major) and out-of-class experiences (first-year 
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students spend more time on formal extracurricular activities while seniors may have studied 
abroad, done internships, and so forth) (Pascarella & Terenzini, 1991). Thus, it would be 
difficult to construct a meaningful institutional typology of engagement from a combined sample 
of both first-year and senior students. 

The overall average unadjusted institutional response rate for NSSE 2001 seniors was 
41.8%. Response rates ranged from 9.1% to 69.7%. About 69% of the seniors completed the 
paper version of the survey, and 31% completed the survey via Web. Generally, administration 
mode does not affect the results, with the exception that web respondents tend to report greater 
use of electronic technology (Carini, Hayek, Kuh, Kennedy, & Ouimet, 2003). Table 2 displays 
the characteristics of NSSE 2001 senior respondents in comparison to the characteristics of all 
seniors at the participating institutions. The results presented in Table 2 indicate that women 
tended to be overrepresented among the respondents, as were Caucasians and full-time students. 
However, the observed differences between respondents and the total population were relatively 
small. 



Insert Table 2 about here 



Measures of Student Engagement 

The NSSE College Student Report asks students to indicate the frequency with which they 
engage in activities that represent good educational practice and are related to positive learning 
outcomes (Kuh et al., 2001). Self-report data is widely used in research on college effects, and 
the validity and credibility of these data have been studied extensively (see Aaker, Kumar, & 



Day, 1998; Baird, 1976; Berdie, 1971; Bradbum & Sudman, 1988; Converse & Presser, 1989; 
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Gershuny & Robinson, 1988; Pace, 1985; Pike, 1995; Pohlmann & Beggs, 1974; Turner & 
Martin, 1984; Wentland & Smith, 1993). Research shows that self-report measures are likely to 
be valid under five conditions: 

(1) the information requested is known to the respondents; 

(2) the questions are phrased clearly and unambiguously; 

(3) the questions refer to recent activities; 

(4) the respondents think the questions merit a serious and thoughtful response; 
and 

(5) answering the question does not threaten, embarrass, or violate the privacy of 
the respondent or encourage the respondent to respond in socially desirable 
ways (Kuh, 2001b, p. 4). 

Studies indicate that the College Student Report meets these five criteria and provides accurate 
and appropriate data about students’ levels of engagement (see Kuh et al., 2001). 

Fifty questions from the College Student Report were summed to create 12 engagement 
scales. These scales were calculated using procedures similar to those used to calculate the 
NSSE benchmarks, and the content of the scales paralleled the content of the benchmarks. For 
example, many of the items comprising the Course Challenge and Student Effort, Writing 
Experiences, and Higher-Order Thinking scales in this study were drawn from the Level of 
Academic Challenge benchmark. Items included in the Active Learning Experiences and 
Collaborative Learning Experiences scales were taken from the Active and Collaborative 
Learning benchmark, and the items comprising the Course-Related Interaction with Faculty and 
Out-of-Class Interaction with Faculty scales were taken from the Student-Faculty Interaction 
benchmark. Similarly, many of the items in the Varied Educational Experiences and Use of 
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Information Technology scales were drawn from the Enriching Educational Experiences 
benchmark. The items included in the Support for Student Success and Interpersonal 
Environment scales came from the Supportive Campus Environment benchmark. The 
Experience with Diversity scale included items from the Active and Collaborative Learning and 
Enriching Educational Experiences benchmarks. Group-mean generalizability analyses 
(Cronbach, Gleser, Nanda, & Rajaratnam, 1972; Kane, Gilmore, & Crooks, 1976; Pike, 1994) 
revealed that dependable (i.e., Ep 2 > 0.70) institutional means could be calculated using as few as 
50 respondents. 

Data Analysis 

The data analysis was conducted in two phases. In the first phase, Q factor analysis was 
used to classify colleges and universities into types of engaging institutions based on similarities 
in their student-engagement profiles. Initially, engagement scale means were calculated for all 
317 colleges and universities in the study. The institutional means were then normalized (i.e., 
transformed to z-scores) to eliminate scaling differences across the 12 student-engagement scales 
and prevent a general species factor from confounding the results (see Cattell, 1952). The data 
matrix was then transposed so that the columns were the 317 institutions and the rows were the 
12 student-engagement scales. Correlations among the 317 institutions were calculated. 

There is no agreement as to whether correlations or distance measures should be used in Q 
factor analysis. Correlations, unlike distances, are measures of pattern similarity and may group 
together institutions that have similar patterns on the 12 engagement scales, but very different 
means (see Guertin, 1971; Rummel, 1970). The advantage of using correlations is that the data 
are ipsatized after they are correlated, and “a normalized-ipsatized data matrix has almost all of 
any general factor eliminated from it” (Gorsuch, 1983, p. 316). 




A Typology of Student Engagement 11 



A principal components analysis, using varimax rotation, was performed using the BMDP 4- 
M program (Dixon, 1992). An analysis using principal axis factoring of a reduced correlation 
matrix with squared multiple correlations in the diagonal did not produce substantively different 
results because the squared multiple correlations were also extremely close to 1.00. The number 
of factors (i.e., types) extracted was determined using eigenvalues, a scree test, and the 
substantive interpretability of the factors or types. Correlations and standard regression 
coefficients for institutions’ normalized engagement scores and factor loadings were used in 
naming the student-engagement types. Institutions with factor loadings greater than or equal to 
0.50 were classified as scoring high on a type, whereas institutions with loadings less than or 
equal to -0.50 were classified as scoring low on a type. Institutions with factor loadings less 
than 0.50, but greater than -0.50, were classified as scoring neither high not low on a type. 

Means on the normalized engagement scales were calculated for these groups and used as an 
additional check in naming the types, 

In the second phase of the data analysis, classifications of institutions as high, low, or 
neither high nor low were cross-tabulated with measures of institutional mission (i.e., Carnegie 
2000 classifications). Chi-square tests were performed, and the contingent proportions in the 
tables were used to interpret relationships between the student-engagement types and Carnegie 
classifications. 

Results 

Student-Engagement Types 

Six factors, representing 80% of the variance in institutional means, were extracted and 
rotated to identify student-engagement types. It is significant that a dominant general factor did 
not emerge from the analyses. The first factor accounted for approximately 21.5% of the 
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variance in institutional means, and the second factor explained 16.5% of the variance. Even the 
sixth factor explained a non-trivial 8.4% of the variance in institutional means. Names of the 
engagement types, eigenvalues, and squared multiple correlations, prior to factor rotation, are 
presented in Table 3. 



Insert Table 3 about here 



Table 4 presents the correlations and standard regression coefficients for institutions’ 
student-engagement means and their loadings on the six rotated factors. Also included are group 
means for institutions classified as high, low, or neither high nor low on the factors. An 
examination of the results in the first subtable reveals that the first factor is bipolar, representing 
two different student engagement types. Institutional means on the Experiences with Diverse 
Groups scale were positively correlated with the factor loadings, whereas Interpersonal 
Environment means were negatively correlated with the factor loadings (0.50 and -0.58, 
respectively). The corresponding betas were 0.56 and -0.36, respectively. Institutional means 
for writing, active learning, and student success also were negatively related to the factor 
loadings, but the strength of these relationships was substantially less than the strength of the 
relationship for the interpersonal environment. Thus, the first factor seems to distinguish 
between institutions characterized by diversity and institutions characterized by positive 
interpersonal relations. The institutions typed by this factor can be characterized as either 
diverse, but interpersonally fragmented or homogeneous and interpersonally cohesive. An 
examination of the group means for institutions classified as either high or low on the first factor 
supports this interpretation. Institutions with high positive loadings on the first factor had a 
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positive mean on the diversity scale (0.70) and a negative mean on the interpersonal environment 
scale (-1.19). Institutions with large negative loadings on the first factor had a positive mean on 
the environment scale (0.66) and a negative mean on the diversity scale (-1.06). 



Insert Table 4 about here 



Two engagement scales, Out-of-Class Interaction with Faculty and Varied Educational 
Experiences, were positively correlated with loadings on the second factor (0.64 and 0.72, 
respectively). The standardized regression coefficients for the two scales also were positive and 
statistically significant (0.45 and 0.61, respectively). Course-Related Interaction with Faculty 
also was positively related to the second factor, although the strength of the relationship was 
much weaker than for the other two scales. Means on the Varied Experiences and Out-of-Class 
Interaction scales for those institutions with high loadings were large and positive (1.51 and 1.62, 
respectively), whereas institutions with low loadings had negative means on the Varied 
Experiences and Out-of-Class Interaction scales (-1.01 and -1.07, respectively). These 
institutions can best be described as intellectually stimulating colleges and universities. 

Three engagement scales, Experiences with Diverse Groups, Support for Student Success, 
and Interpersonal Environment, were positively correlated with loadings on the third factor 
(0.36, 0.43, and 0.43, respectively). Standardized regression coefficients were also positive 
(0.51, 0.43, and 0.37, respectively). Institutions with high loadings on the third factor had 
positive means for the diversity (0.60), support (0.73), and environment scales (0.74), whereas 
institutions with low factor loadings had negative scale means (-0.98, -0.76, and -0.53, 
respectively). These colleges and universities are interpersonally supportive institutions. 
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The correlations, regression coefficients, and group means for the fourth factor suggested 
that the underlying construct for this factor was engagement through information technology. 

The correlation between Use of Information Technology scores and factor loadings was 0.59, 
and the standardized regression coefficient was 0.82. The information technology scale mean for 
institutions with high loadings was 0.85, and the mean for institutions with low loadings on the 
factor was -1.29. The institutions typed by this factor are high-tech, low-touch universities. 

The fifth factor represented institutions that were academically challenging and supportive. 
These institutions has high levels of academic challenge and student effort. The correlation 
between institutional means on the Course Challenge and Student Effort scale and loadings on 
the fifth factor was 0.58. The standardized regression coefficient was 0.71. Institutions with 
high loadings on the fifth factor had a mean course challenge score of 1.81, and institutions with 
low loadings had a mean score of -0.86. 

The results in the final subtable indicated that the sixth factor represented institutions with 
high levels of collaborative learning. The Collaborative Learning Experiences scale was 
positively correlated with loadings on the sixth factor (0.52), and the standardized regression 
coefficient was 0.80. The group mean for institutions with high loadings on the sixth factor was 
1.47, whereas the mean for institutions with low loadings was -1.50. These institutions were 
labeled collaborative in this study. 

Engagement Types and Carnegie Classifications 

Chi-square tests of the relationships between engagement types and Carnegie classifications 
indicated that the different types of engaging institutions were related to differences in 
institutional missions; although the relationships generally did not support conventional wisdom 
that small, private liberal arts colleges have the highest levels of engagement. The relationships 
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between engagement types and Carnegie classifications are presented in Table 5. Chi-square 
tests revealed that loadings on the first factor were significantly related to Carnegie 
classifications (x = 109.43; df = 10; p < 0.001). An examination of the cross-tabulation of the 
first two engagement types in the first subtable and Carnegie classifications shows that diverse, 
but interpersonally fragmented, institutions tended to be Doctoral/Research-Extensive, and to a 
lesser extent Doctoral/Research-Intensive, universities. All other Carnegie classes were 
underrepresented in this engagement type. Conversely, interpersonally cohesive colleges and 
universities tended to be Masters I and II institutions. Follow-up analyses revealed that the 
institutions with positive interpersonal environments tended to be private (70.2%) and have 
enrollments of less than 3,000 students (49.6%). 



Insert Table 5 about here 



Chi-square tests also indicated that institutions’ loadings on the second factor were related to 
their Carnegie classifications (~/_ 2 = 65.76; df = 10; p < 0.001). The contingent proportions in the 
second subtable indicated that intellectually stimulating colleges and universities tended to be 
smaller Baccalaureate-Liberal Arts institutions. Chi-square results indicated that there was not a 
significant relationship between Carnegie classifications and interpersonally supportive 
institutions (x 2 = 15.38; df = 10; p > 0.05). In contrast, there was a statistically significant 
relationship between Carnegie classifications and high-tech, low-touch institutions (% 2 = 52.65; 
df = 10; p < 0.001). Institutions characterized by high levels of engagement using information 
technology tended to be Doctoral/Research-Extensive universities. 
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Chi-square tests also revealed that there was a statistically significant relationship between 
loadings on the fifth factor and Carnegie classifications (% 2 = 33.01; df = 10; p < 0.001). 
Specifically, academically challenging and supportive colleges and universities tended to be 
baccalaureate institutions, both Baccalaureate-Liberal Arts and Baccalaureate-General colleges. 
Most were private (73.9%) and 86.9% had enrollments of under 3,000 students. There was also 
a statistically significant relationship between institutions’ loadings on the final factor and their 
Carnegie classifications (% 2 = 32.30; df = 10; p < 0.001). Contingent proportions revealed that 
Masters I and, to a lesser extent, Doctoral/Research-Intensive institutions tended to have high 
levels of engagement in collaborative learning experiences. Furthermore, these collaborative 
colleges and universities tended to be public institutions (66.7%) and 
have enrollments ranging from 3,000 to 10,000 students. 

Limitations 

While results are generally consistent with the results reported by NSSE across the first few 
years of its surveys, only one year of data was analyzed in this study. If more institutions 
participating in other years were included, the results might differ in unknown ways. The 
institutional categories were derived only from the responses of seniors. If a similar analysis was 
done using first-year students, different factors may have resulted. Also, the NSSE survey is 
relatively short and, as a result, some potentially positive educationally purposeful activities are 
not represented, such as experiences in residence halls, the performing arts, and so on. If 
questions related to these activities were included, perhaps different results would emerge. Also, 
if multiple institutional characteristics were employed in the analysis, such as percentage of 
students in different majors or a measure of students’ socioeconomic status, different 
institutional types might be produced. Finally, while Q factor analysis is an accepted 
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methodology for constructing typologies, other methods could produce different categories of 
institutions. Perhaps most important, Q factor analysis is a correlational procedure, and the chi- 
square analyses provided measures of association. Consequently, the findings of this study are 
descriptive and do not imply causal relationships. Despite these limitations, the results of the 
present research do have important implications for theory and practice. 

Discussion 

This study examined whether it was possible to identify different institutional patterns of 
engagement that were not based on Carnegie classifications. First and foremost, this study 
identified seven types of engaging institutions: 

• Diverse, but Interpersonally Fragmented. Students at these colleges have numerous 
experiences with diversity and tend to use technology, but do not view the institution as 
supporting their academic or social needs nor are their peers viewed as supportive or 
encouraging. All in all, not a very easy place to live and learn it seems. 

• Homogeneous and Interpersonally Cohesive. Students at these colleges have relatively 
few experiences with diversity, but view the institution and their peers as supportive. 
These institutions are the mirror image of the first engagement type. 

• Intellectually Stimulating. Students at these colleges are engaged in a variety of 
academic activities and have a great deal of interaction with faculty inside and outside 
the classroom. The also tend to engage in higher-order thinking and work with their 
peers on academic matters (i.e., collaborative learning). 

• Interpersonally Supportive. Students attending these institutions report high frequency 
of diversity experiences and view their peers and the campus as supportive of their 
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efforts. Students also have a reasonable amount of contact with faculty members inside 
and outside the classroom. 

• High-Tech, Low-Touch. Information technology rules at these universities to the point 
of muting other types of interactions. There is a sense of stark individualism as little 
collaboration occurs, academic challenge is low, and the interpersonal environment is 
not a distinguishing feature of the campus. 

• Academically Challenging and Supportive. Faculty set high expectations and emphasize 
higher-order thinking in traditional ways. Little active and collaborative learning is 
required. At the same time, students support one another and view the campus as 
supportive. A generally friendly and congenial place to be an undergraduate interested in 
learning. 

• Collaborative. Peers rely on and are generally supportive of one another for learning, 
mediated somewhat by technology. Although there are few opportunities for 
experiences with diversity, students have a reasonable amount of contact with faculty, 
who along with other dimensions of the campus climate, are viewed as supportive. 

The results of this research also confirmed the observations made in the first four NSSE 
national reports — that institutions differ in how they engage students and that no institution is 
uniformly high or low across all measures of engagement. At the same time, engagement and 
Carnegie classification was related. However, counter to conventional wisdom, institutional type 
and engagement did not favor liberal arts colleges and universities. While many of the 
institutions with high levels of engagement in varied educational experiences, interaction with 
faculty outside of class, and course challenge and student effort were baccalaureate colleges and 
universities, other types of engagement were associated with other Carnegie classifications. For 
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example, large public research universities had higher than expected levels of engagement with 
diverse groups of students and high levels of engagement through information technology. 
Master’s institutions were most likely to be characterized by positive interpersonal environments, 
support for student success, and high levels of collaborative learning. 

Even though Carnegie type was related in non-trivial ways to certain areas of engagement, it 
is both possible and probable that other institutional characteristics are shaping engagement in 
addition to the gross measure of mission that Carnegie purports to emphasize. For example, that 
students use electronic technology more frequently at doctoral extensive universities may be 
more a function of institutional investment in technology than other structural features of such 
universities. If smaller comprehensive universities made similar investments they, too, could be 
technology intensive in terms of student engagement. Similarly, there is nothing inherent in the 
structural elements of campus environments of masters-granting colleges and universities that 
explain why students find them more supportive overall of their educational endeavors. Indeed, 
of the various Carnegie types, these institutions may be among the more eclectic in terms of size, 
educational purposes, and other dimensions. Thus, efforts to understand how institutional 
mission and environments shape student engagement will have to take into account additional 
factors in their analyses if student engagement is to be explained by institutional characteristics. 

The results from this study also indicate that the 12 engagement scales used in this 
study can serve as useful metrics for institutional assessment. The 12 scales were particularly 
effective at differentiating among institutions and among types of engagement. In fact, the 
relationships between the 12 engagement scales and the different types of engaging institutions 
are revealing. Some institutions were engaging in a single domain (e.g., challenging coursework, 
collaborative learning, or use of information technology), whereas other institutions engaged 
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students across several domains (e.g., varied educational experiences and out-of-class interaction 
or experiences with diverse groups of students and a positive interpersonal environment). 
Moreover, all of the scales produced acceptable generalizability coefficients (0.70) with samples 
of as few as 50 students. When samples sizes approached 200 students, the group-mean 
generalizability coefficients were all in excess of 0.90. These levels of generalizability should 
allow many institutions to examine levels of engagement at the college or school level. 

Among the more promising lines of using assessment for institutional improvement is to 
obtain reliable data at the major field or department level; faculty members are more likely to 
take responsibility for student engagement if they are certain the data represent “their” students. 
Thus, institutional researchers and assessment staff should consider ways to administer student 
engagement surveys and other tools that insure that a reasonable number of majors in a given 
area are among the respondents, which will allow disaggregation at the department level. 

Another approach is to collect data about student engagement directly from faculty, such as 
using the Faculty Survey of Student Engagement (http://www.iub.edu/~nsse/html/fsse.htm) . 
Responses can then be compared from students and faculty members to determine where gaps 
exist in what faculty members expect and assume students are doing and what students actually 
do. Indeed, perhaps what appear to be institutional differences in student engagement that favor 
one type of institution over others on certain engagement dimensions are more a function of what 
faculty members require students to do rather than institutional characteristics. 

Although the emergence in this study of different types of engaging institutions is 
encouraging, the relationship between interactions with diverse groups and the quality of the 
interpersonal environment at some institutions was disturbing. Recent court decisions have 
upheld preferential admission policies based on the premise that attracting a more diverse student 
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body will lead to greater interaction among the groups, and ultimately lead to more positive 
relations among students (Hurtado, Dey, Gurin, & Gurin, 2003).. This study provided support 
for the first part of the premise that diverse student populations lead to interaction with diverse 
groups. A follow-up analysis indicated that the institutions with high levels of diversity 
engagement tended to be from parts of the country with substantial minority populations and the 
institutions had relatively large minority enrollments. 

It is the second part of the premise that was not uniformly supported in this study. For 
several institutions with high levels of engagement around diversity, the interpersonal 
environment was reported to be negative. Instead, the most supportive interpersonal 
environments were found in institutions characterized by little interaction with diverse groups. 
Relatively few institutions had both high levels of diversity interaction and positive 
interpersonal environments. This finding provides a sharp contrast to the NSSE results reported 
by Umbach and Kuh (2003). 

The findings of the current study may indicate that the tone of interactions among 
groups is a key factor influencing the interpersonal environment at an institution. When 
interactions among diverse groups are positive, perceptions of the interpersonal environment 
are likely to be positive, whereas negative interactions among groups may lead to perceptions 
of the campus environment that are negative (Hurtado, Dey, Gurin, & Gurin, 2003). As a 
result, efforts to improve acceptance and appreciation of diversity may need to do much more 
than attract diverse students to campus and foster interaction among students. Interactions 
among diverse groups are most likely to have positive effects when the groups are of equal 
status, there are common goals and inter-group cooperation, authorities support group equality, 
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and there are extended opportunities for group members to get to know one another (see 
Allport, 1954; Hurtado, Dey, Gurin, & Gurin, 2003). 

It is also possible that the association between interactions among diverse groups of students 
and negative perceptions of the interpersonal environment is spurious. Many of the institutions 
comprising the first engagement type are large research universities with reputations as 
academically rigorous institutions. High levels of diversity experiences and less-than-satisfying 
interpersonal relations may both accrue from the size and competitiveness of these institutions, 
rather than being directly and causally related. 

Clearly, additional research is need to better understand the relationships among diversity of 
the student body, interactions among diverse groups of students, and the interpersonal 
environments at an institution. It would be especially instructive to learn more about the 
colleges and universities where students both report high levels of diversity experiences and 
perceive the interpersonal environment to be supportive. Such institutions may have in place 
policies and practices that could be adapted by other schools to improve the overall quality of 



the undergraduate experience. 
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Table 1 



Characteristics of Institutions Participating in NSSE 2001 





NSSE 2001 


National 


Carnegie Classification 


Doc/Res - Ext 


16% 


11% 


Doc/Res - Int 


10% 


8% 


Master’s I & II 


42% 


43% 


Bac - Liberal Arts 


21% 


16% 


Bac - General 


11% 


23% 


Sector 


Public 4- Year 


48% 


36% 


Private 4- Year 


52% 


64% 


Region 


Far West 


9% 


10% 


Great Lakes 


20% 


16% 


Mideast 


19% 


19% 


New England 


9% 


9% 


Plains 


8% 


11% 


Rocky Mountains 


3% 


3% 


Southeast 


22% 


26% 


Southwest 


9% 


7% 


Location 


Large City 


20% 


19% 


Mid-Size City 


32% 


29% 
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Urban Fringe Large City 


17% 


17% 


Urban Fringe Small City 


7% 


8% 


Large Town 


5% 


4% 


Small Town 


13% 


17% 



Rural 



5% 



6% 
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Table 2 

Characteristics of NSSE 2001 Senior Respondents 





NSSE 

Respondents 


NSSE 2001 
Schools 


Gender 


Men 


35.4% 


41.9% 


Women 


64.5% 


58.1% 


Race/Ethnicity 


African American/Black 


5.8% 


8.0% 


American Indian/ Alaska Native 


0.6% 


0.7% 


Asian/Pacific Islander 


5.3% 


4.2% 


Caucasian/White 


79.0% 


75.4% 


Hispanic 


4.2% 


5.8% 


Other 


0.2% 


2.9% 


Multiple 


4.8% 


NA 


International 


4.0% 


3.0% 


Enrollment Status 


Full-Time 


83.1% 


77.7% 


Part-Time 


16.9% 


22.3% 
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Table 3: 



Eigenvalues and Squared Multiple Correlations for the Factors 



Factor 


Eigenvalue 


SMC 


1 


68.24 


0.215 


2 


52.23 


0.165 


3 


40.75 


0.128 


4 


35.43 


0.112 


5 


29.91 


0.095 



6 



26.69 



0.084 
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Table 4: 

Correlations, Regression Coefficients, and Group Means 



Diverse but Interpersonally 
Fragmented vs. Homogeneous and 
Interpersonally Cohesive 





Correla- 


Std. Reg. 


Low 


Neither 


High 


Scale 


tion 


Coeff. 


Mean 


Mean 


Mean 


Course Challenge 


0.01 


0.16 


-0.34 


0.07 


-0.21 


Writing 


-0.41 


-0.22 


0.50 


0.08 


-0.96 


Higher-Order Thinking 


0.04 


0.15 


-0.31 


0.06 


-0.15 


Active Learning 


-0.46 


-0.28 


0.59 


0.08 


-1.04 


Collaborative Learning 


-0.09 


0.01 


0.05 


0.06 


-0.45 


Course Interaction 


-0.16 


0.12 


0.10 


0.07 


-0.55 


Out-of-Class Interaction 


-0.23 


-0.18 


0.34 


0.05 


-0.63 


Varied Experiences 


-0.18 


-0.09 


0.20 


0.03 


-0.43 


Information Technology 


0.38 


0.25 


-0.49 


-0.03 


0.58 


Diversity 


0.50 


0.56 


-1.06 


0.02 


0.70 


Support Success 


-0.44 


-0.17 


0.51 


0.08 


-0.98 


Interpersonal Environment 


-0.58 


-0.36 


0.66 


0.10 


-1.19 


Intellectually Stimulating 


Correla- 


Std. Reg. 


Low 


Neither 


High 


Scale 


tion 


Coeff. 


Mean 


Mean 


Mean 


Course Challenge 


-0.11 


-0.39 


-0.15 


0.03 


-0.28 


Writing 


-0.04 


-0.26 


-0.02 


0.02 


-0.21 


Higher-Order Thinking 


0.30 


0.22 


-0.79 


0.02 


0.24 



Active Learning 



0.01 



-0.18 



-0.19 



0.02 



-0.16 
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Collaborative Learning 


0.20 


-0.10 


-0.54 


0.01 


0.15 


Course Interaction 


0.41 


0.26 


-1.10 


0.01 


0.47 


Out-of-Class Interaction 


0.64 


0.45 


-1.51 


-0.01 


1.01 


Varied Experiences 


0.72 


0.61 


-1.62 


-0.01 


1.07 


Information Technology 


0.30 


-0.06 


-0.64 


0.02 


0.14 


Diversity 


0.11 


-0.14 


-0.30 


0.04 


-0.28 


Support Success 


0.17 


-0.03 


-0.68 


0.03 


0.07 


Interpersonal Environment 


-0.07 


-0.27 


-0.29 


0.05 


0.42 


Intcrnersonally Supportive 














Correla- 


Std. Reg. 


Low 


Neither 


High 


Scale 


tion 


Coeff. 


Mean 


Mean 


Mean 


Course Challenge 


0.08 


-0.12 


-0.23 


0.03 


-0.30 


Writing 


-0.24 


-0.54 


0.43 


0.02 


-0.64 


Higher-Order Thinking 


-0.04 


-0.22 


-0.12 


0.03 


-0.43 


Active Learning 


-0.10 


0.02 


-0.28 


0.02 


0.02 


Collaborative Learning 


-0.20 


-0.22 


0.23 


0.02 


-0.52 


Course Interaction 


0.20 


0.25 


-0.42 


0.01 


0.19 


Out-of-Class Interaction 


0.11 


0.07 


-0.31 


0.02 


0.02 


Varied Experiences 


-0.08 


-0.21 


0.07 


0.00 


-0.09 


Information Technology 


-0.37 


-0.36 


0.54 


0.03 


-0.87 


Diversity 


0.36 


0.51 


-0.98 


0.02 


0.60 


Support Success 


0.43 


0.43 


-0.76 


0.00 


0.73 


Interpersonal Environment 


0.43 


0.37 


-0.53 


-0.02 


0.74 


Hiuh-Tech, Low-Touch 














Correia- 


Std. Reg. 


Low 


Neither 


High 
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Scale 


tion 


Coeff. 


Mean 


Mean 


Mean 


Course Challenge 


-0.43 


-0.31 


0.72 


0.02 


-0.92 


Writing 


-0.25 


-0.01 


0.45 


0.01 


-0.60 


Higher-Order Thinking 


-0.34 


-0.24 


0.80 


0.00 


-0.79 


Active Learning 


-0.48 


-0.24 


0.91 


0.01 


-1.03 


Collaborative Learning 


-0.26 


-0.28 


0.66 


-0.01 


-0.48 


Course Interaction 


-0.23 


-0.12 


0.42 


0.01 


-0.61 


Out-of-Class Interaction 


-0.12 


-0.05 


0.14 


0.03 


-0.56 


Varied Experiences 


-0.04 


0.10 


0.02 


0.02 


-0.33 


Information Technology 


0.59 


0.82 


-1.29 


0.02 


0.85 


Diversity 


-0.22 


-0.13 


0.47 


0.01 


-0.52 


Support Success 


-0.16 


0.22 


0.20 


0.01 


-0.40 


Interpersonal Environment 


-0.06 


0.25 


-0.28 


0.04 


-0.34 


Academically Challenging and 
Supportive 


Correla- 


Std. Reg. 


Low 


Neither 


High 


Scale 


tion 


Coeff. 


Mean 


Mean 


Mean 


Course Challenge 


0.58 


0.71 


-0.86 


-0.06 


1.81 


Writing 


-0.05 


-0.16 


0.08 


-0.02 


0.28 


Higher-Order Thinking 


0.26 


0.36 


-0.35 


-0.04 


1.01 


Active Learning 


-0.17 


-0.41 


0.43 


-0.01 


-0.09 


Collaborative Learning 


-0.29 


-0.50 


0.56 


0.01 


-0.58 


Course Interaction 


0.10 


0.03 


-0.11 


-0.02 


0.61 


Out-of-Class Interaction 


0.01 


-0.02 


0.12 


-0.02 


0.36 


Varied Experiences 


0.11 


0.08 


-0.10 


-0.04 


0.95 
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Information Technology 


-0.15 


-0.12 


0.19 


0.00 


-0.06 


Diversity 


-0.11 


-0.34 


0.29 


0.00 


-0.13 


Support Success 


0.28 


0.13 


-0.48 


-0.02 


0.80 


Interpersonal Environment 


0.31 


0.20 


-0.46 


-0.02 


0.76 


Collaborative 














Correla- 


Std. Reg. 


Low 


Neither 


High 


Scale 


tion 


Coeff. 


Mean 


Mean 


Mean 


Course Challenge 


0.14 


0.24 


-0.60 


0.00 


0.29 


Writing 


-0.26 


-0.53 


-0.16 


0.02 


-0.55 


Higher-Order Thinking 


-0.12 


-0.27 


-0.42 


0.01 


0.05 


Active Learning 


-0.15 


-0.24 


-0.75 


0.02 


-0.32 


Collaborative Learning 


0.52 


0.80 


-1.50 


-0.01 


1.47 


Course Interaction 


0.11 


0.06 


-0.77 


0.01 


0.11 


Out-of-Class Interaction 


0.23 


0.22 


-1.10 


0.01 


0.34 


Varied Experiences 


0.00 


-0.23 


-0.65 


0.01 


-0.19 


Information Technology 


0.23 


0.14 


-0.82 


0.00 


0.47 


Diversity 


-0.27 


-0.41 


-0.35 


0.03 


-0.89 


Support Success 


0.14 


0.13 


-0.81 


0.01 


0.23 


Interpersonal Environment 


0.13 


0.18 


-0.74 


0.01 


0.31 
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Table 5: 



Relationships Between Engagement Types and Carnegie Classifications 



Diverse but Intcrnersonally 
Fragmented vs. Homogeneous and 
Internersonallv Collaborative 


High 


Neither 


Low 


Total 


Doc/Res-Ext 


58.7% 


9.8% 


0.0% 


15.5% 


Doc/Res-Int 


17.4% 


11.2% 


0.0% 


10.4% 


Masters I 


19.6% 


37.5% 


55.3% 


37.5% 


Masters II 


0.0% 


4.5% 


12.8% 


5.0% 


Bac-Lib Arts 


4.3% 


26.3% 


12.8% 


21.1% 


Bac-General 


0.0% 


10.7% 


19.1% 


10.4% 


Intellectually Stimulating 












High 


Neither 


Low 


Total 


Doc/Res-Ext 


11.5% 


18.4% 


0.0% 


15.5% 


Doc/Res-Int 


0.0% 


11.2% 


12.2% 


10.4% 


Masters I 


15.4% 


35.6% 


63.4% 


37.5% 


Masters II 


0.0% 


6.0% 


2.4% 


5.0% 


Bac-Lib Arts 


73.1% 


18.0% 


7.3% 


21.1% 


Bac-General 


0.0% 


10.8% 


14.6% 


10.4% 


Internersonallv Supportive 












High 


Neither 


Low 


Total 


Doc/Res-Ext 


0.0% 


16.9% 


18.5% 


15.5% 


Doc/Res-Int 


6.7% 


10.0% 


18.5% 


10.4% 


Masters I 


46.7% 


35.4% 


48.1% 


37.5% 
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Masters II 


10.0% 


4.6% 


3.7% 


5.0% 


Bac-Lib Arts 


23.3% 


21.9% 


11.1% 


21.1% 


Bac-General 


13.3% 


11.2% 


0.0% 


10.4% 


High-Tech, Low-Touch 


High 


Neither 


Low 


Total 


Doc/Res-Ext 


54.8% 


12.6% 


0.0% 


15.5% 


Doc/Res-Int 


9.7% 


11.4% 


3.1% 


10.4% 


Masters I 


22.6% 


36.2% 


62.5% 


37.5% 


Masters II 


3.2% 


5.5% 


3.1% 


5.0% 


Bac-Lib Arts 


6.5% 


23.2% 


18.8% 


21.1% 


Bac-General 


3.2% 


11.0% 


12.5% 


10.4% 


Academically Challenging and 
Supportive 


High 


Neither 


Low 


Total 


Doc/Res-Ext 


8.7% 


16.4% 


12.0% 


15.5% 


Doc/Res-Int 


4.3% 


10.8% 


12.0% 


10.4% 


Masters I 


0.0% 


39.8% 


48.0% 


37.5% 


Masters II 


4.3% 


5.2% 


4.0% 


5.0% 


Bac-Lib Arts 


56.5% 


19.0% 


12.0% 


21.1% 


Bac-General 


26.1% 


8.9% 


12.0% 


10.4% 


Collaborative 


High 


Neither 


Low 


Total 


Doc/Res-Ext 


11.1% 


15.9% 


14.3% 


15.5% 


Doc/Res-Int 


18.5% 


7.6% 


50.0% 


10.4% 


Masters I 


48.1% 


37.0% 


28.6% 


37.5% 
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Masters II 


3.7% 


5.4% 


0.0% 


5.0% 


Bac-Lib Arts 


14.8% 


22.5% 


7.1% 


21.1% 


Bac-General 


3.7% 


11.6% 


0.0% 


10.4% 



Doc/Res-Ext = Doctoral/Research-Extensive, Doc/Res-Int = Doctoral/Research-Intensive, Bac- 
Lib Arts = Baccalaureate Liberal Arts, Bac-General = Baccalaureate General 





